The rapid development of commercial image search engines has allowed users to easily retrieve a large number of images simply by typing in a text query into a search engine. Existing search engines, however, only use the relevance of text information associated with images in ranking image search results. Existing search engines typically do not use the visual information associated with images in providing search results.
The growth of digital image content has made it more of a challenge to browse through a large number of search results. Two techniques commonly employed to assist with search result refinement are content-based re-ranking and a technique called IntentSearch. Content-based re-ranking relies on image clustering and categorization to provide a high-level description of a set of images. While content-based re-ranking uses visual information to reorder the search results, it does not take into consideration the intent of the user. On the other hand, IntentSearch provides an interface to allow users to indicate a few images of interest, and automatically attempts to guess the intent of the user to reorder image search results. However, guessing the intent of the user is somewhat difficult based on selected images. Recently, a color structured image search was proposed to enable the user to indicate their intent by simply drawing a few color strokes on a blank image that reflects the color spatial distribution that the user is looking for in an image. With this technique, it is not easy for a user to indicate their semantic intent or the spatial distribution of the content they are seeking in an image.